Skip to content
This repository has been archived by the owner on Aug 12, 2024. It is now read-only.

feat: move system Flatpak handling out of Yafti #88

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

cassidyjames
Copy link
Collaborator

@cassidyjames cassidyjames commented Jul 30, 2023

This takes system Flatpak handling out of Yafti and does it the same way as we tried on endlish-oesque.

  • Flathub is included in the image as a system remote the way the Flatpak documentation recommends
  • /etc/flatpak/install contains a list of Flatpaks to be installed
  • A new line in Containerfile reads /etc/flatpak/install and installs the apps without deploying (to preseed the needed files)
  • ublue-flatpak-system-install reads /etc/flatpak/install and installs what's there without pulling remote data
  • flatpak-system-install systemd unit runs ublue-flatpak-system-install at boot

This takes system Flatpak handling out of Yafti and does it the same way as we tried on endlish-oesque.
It adds an /etc/flatpak/install file with a list of Flatpaks to be installed, a step in the Containerfile
to read that file and install the apps without deploying (to preseed the needed files), a
ublue-flatpak-system-install script to read that file and install what's there without pulling remote
data, and a systemd unit to run that script at boot.
@cassidyjames cassidyjames requested a review from castrojo as a code owner July 30, 2023 08:35
@cassidyjames cassidyjames changed the title feat: move system Flatpak handling to reflect endlish-oesque method feat: move system Flatpak handling out of Yafti Jul 30, 2023
Copy link
Member

@p5 p5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is clean!!! Might steal this for my own repos 😁

I guess installing the system flatpaks are much faster after storing the necessary files in the image.

I hope something similar can be used to cache the user level flatpaks.

@cassidyjames
Copy link
Collaborator Author

cassidyjames commented Jul 30, 2023

@p5 and importantly, you don't require the Internet at boot. 😄 This was the impetus for doing it that way in endlish-oesque, as that's a major part of Endless OS.

@@ -21,9 +21,6 @@ screens:
- run: /usr/lib/fedora-third-party/fedora-third-party-opt-out
- run: /usr/bin/fedora-third-party disable
- run: flatpak remote-delete fedora --force
- run: flatpak remove --system --noninteractive --all
Copy link
Member

@p5 p5 Jul 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious - how do you plan to remove the pre-installed Fedora Flatpaks when rebasing from an existing Silverblue install? Wouldn't you end up with duplicate flatpaks from different remotes?
Or are you assuming this will be installed via ISOs - therefore not contain existing Flatpaks?

Perhaps this flatpak remove step is best added to the systemd oneshot service? Bad idea

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@p5 yeah I wondered about this as well; doesn't the line immediately above handle that?

But it should definitely be tested. I didn't remove this line in endlish-oesque (https://github.com/ublue-os/endlish-oesque/blob/36f22545c9e1cac3cafb45848fb022067b21c9a1/etc/yafti.yml#L20-L26) and I recall some issues with the system remote being removed and re-added unnecessarily.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could copy this back into endlish-oesque and just see what happens when rebasing from a stock Fedora image, as it's an even more experimental image and I don't mind pushing broken things there. 😄

Copy link
Member

@p5 p5 Jul 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to update your script to add logic to remove Fedora flatpaks if the oneshot hasn't been ran already. Will let you know if it works
https://github.com/rsturla/eternal-images/pull/21/files#diff-cac88664baf2bb1d4c843b4247ea9ab0ccaa0b13b00cf55a5c63768f0f4dd97eR1

doesn't the line immediately above handle that?

I believe the line you're referring to only removes the remote, so you no longer get updates from the Fedora repos and cannot install new Fedora flatpaks. I don't think it deletes any of the packages. I may be wrong though, so don't blindly take my word for it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@p5 ah you're right; I wasn't aware a Flatpak could get orphaned like that. I'll poke around a bit with the Flatpak CLI and docs. 😄

Copy link
Collaborator Author

@cassidyjames cassidyjames Jul 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean rather than removing all, I could just specifically target the Fedora remote. Duh! 🤦🏻

Flatpak has limited CLI when it comes to removing apps from a specific remote as opposed to a specific installation. Hrm.

Copy link
Member

@p5 p5 Jul 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has this method of installing Flatpaks been confirmed in Silverblue OCI images?
It looks like files are stored in /var/lib/, which is not allowed by ostree.

98.73 Found file: var/lib/flatpak/repo/objects/4b/54aceab94ad094bbc1154f2f160a77b27c3f339caf57c1cfb7da33b9ce534d.file
98.73 Found file: var/lib/flatpak/repo/objects/4b/b51a1d2fb4dbb5eda84f52cd47862efbb4c62f03d27512e3f6c66c3df8c0de.file
98.91 error: Found content in var

It looks like you're clearing out that directory at the end of your layer, but that cannot happen if the cache/artefacts are stored in that directory.

Where does Flatpak store the artefacts from --no-deploy?

Copy link
Member

@p5 p5 Jul 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh, you can change the cache directory using the following environment variable

FLATPAK_SYSTEM_CACHE_DIR

https://docs.flatpak.org/en/latest/flatpak-command-reference.html#Environment

We will likely need to set that to somewhere else since it defaults to /var/tmp, which must be clear when running ostree container commit.

Note: This is only required if --no-deploy stores the files in /var/, which I'm not sure about

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it'd be fine to add this line to the Containerfile (or a script called by it) so that they get removed while the image is built

@@ -0,0 +1,8 @@
[Flatpak Repo]
Copy link
Member

@EyeCantCU EyeCantCU Jul 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious - can the repo be added via adding:

flatpak remote-add --system --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak remote-modify --system flathub --no-filter --title="Flathub (System)"

To the Containerfile? Love this idea by the way

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes to the first command.
Not tried the second

Copy link
Member

@EyeCantCU EyeCantCU Jul 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. Went ahead and gave it a shot, and this ran without failure (the script gets read from my recipe and called in the build script called by the Containerfile): EyeCantCU/Atlas@1328112 (the title of this commit is misleading, I was in a hurry and amended the prior commit)

Throwing this into my Containerfile also seems to have worked:

RUN flatpak remote-add --system --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo && \
    flatpak remote-modify --system flathub --no-filter --title="Flathub (System)"

Either way, it's awesome to hear the remote can at least be added that way

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see why manually including this is needed now. My instructions above only work for the build process. The remote never gets added to the users system the way I set it up

@EyeCantCU
Copy link
Member

Hi!

I managed to get this working over on Bazzite with some work, and I'd like to share my findings

Installing all the flatpaks without deploying them works fine, but they aren't cached nicely on the image. We need /var/lib/flatpak/repo/objects and /var/lib/flatpak/repo/remotes/flathub, but since var must be empty when the image is created, we have to move them somewhere else. I know it's ugly but I opted for that to be /etc/flatpak/flathub and /etc/flatpak/objects. These files are not cached by flatpak in the directory defined by FLATPAK_SYSTEM_CACHE_DIR so it has to be done this way (unless there are any better suggestions on this one, I'd really like to know)

On initial boot, I configured the service to move these files back over to /var/lib/flatpak/.. and to remove the directories so that space isn't being wasted. I also opted to check for the existence of /etc/flatpak/install before running the script as otherwise the service will fail after the file has been moved

https://github.com/ublue-os/bazzite/pull/86/files

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants